A Unified Approach to Model Selection and Sparse Recovery Using Regularized Least Squares1 by Jinchi
نویسنده
چکیده
Model selection and sparse recovery are two important problems for which many regularization methods have been proposed. We study the properties of regularization methods in both problems under the unified framework of regularized least squares with concave penalties. For model selection, we establish conditions under which a regularized least squares estimator enjoys a nonasymptotic property, called the weak oracle property, where the dimensionality can grow exponentially with sample size. For sparse recovery, we present a sufficient condition that ensures the recoverability of the sparsest solution. In particular, we approach both problems by considering a family of penalties that give a smooth homotopy between L0 and L1 penalties. We also propose the sequentially and iteratively reweighted squares (SIRS) algorithm for sparse recovery. Numerical studies support our theoretical results and demonstrate the advantage of our new methods for model selection and sparse recovery.
منابع مشابه
A Unified Approach to Model Selection and Sparse Recovery
Model selection and sparse recovery are two important problems for which many regularization methods have been proposed. We study the properties of regularization methods in both problems under the unified framework of regularized least squares with concave penalties. For model selection, we establish conditions under which a regular-ized least squares estimator enjoys a nonasymptotic property,...
متن کاملFace Recognition using an Affine Sparse Coding approach
Sparse coding is an unsupervised method which learns a set of over-complete bases to represent data such as image and video. Sparse coding has increasing attraction for image classification applications in recent years. But in the cases where we have some similar images from different classes, such as face recognition applications, different images may be classified into the same class, and hen...
متن کاملGroup Sparse Recovery via the ℓ0(ℓ2) Penalty: Theory and Algorithm
In this work we propose and analyze a novel approach for recovering group sparse signals, which arise naturally in a number of practical applications. It is based on regularized least squares with an `(`) penalty. One distinct feature of the new approach is that it has the built-in decorrelation mechanism within each group, and thus can handle the challenging strong inner-group correlation. We ...
متن کاملSparse Representation for Detection of Microcalcification Clusters
We present an approach to detect MCs in mammograms by casting the detection problem as finding sparse representations of test samples with respect to training samples. The ground truth training samples of MCs in mammograms are assumed to be known as a priori. From these samples of the interest object class, a vocabulary of information-rich object parts is automatically constructed. The sparse r...
متن کاملNetwork Intrusion Detection through Discriminative Feature Selection by Using Sparse Logistic Regression
Intrusion detection system (IDS) is a well-known and effective component of network security that provides transactions upon the network systems with security and safety. Most of earlier research has addressed difficulties such as overfitting, feature redundancy, high-dimensional features and a limited number of training samples but feature selection. We approach the problem of feature selectio...
متن کامل